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ABSTRACT 



First-year evaluation of the Word Attack component of 
the Wisconsin Design for Reading was conducted in 23 elementary 
schools representing disadvantaged inner-city and advantaged schools 
in Wisconsin and Colorado- Primary-grade pupils in each of the 
schools experienced the program for 6 months, during which 
instruction was followed by criterion-referenced tests to indicate 
individual pupil mastery of skills- Subjects in all schools who had 
not experienced the program were tested in May, 1970, using the 
Wisconsin Tests of Reading Skill Development- — Word Attack and three 
standardized reading tests- Subjects who had experienced the program 
were given the same tests in May, 1971- Simple t— tests and complex 
variations of t^tests were used to analyze the data- Subjects who had 
participated in the program achieved significantly higher scores on 
the standardized tests of reading, vocabulary, and comprehension, and 
on tests associated with the specific program objectives. The results 
of the first-year field testing and evaluation of the Word Attack 
Component were judged to be encouraging. References are included. 
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The formal field test of the Word Attack began during the 1970-71 
school year and is continuing during the 197 1-72 school year* The Study 
Skills and Comprehension components will be field tested at a later time 
and will not reported in this paper. 



*This is the fifth paper in a symposium *■- A Skill Centered Approach to 
Reading Instruction! The Wisconsin Design for Reading Skill Development 
-- at the 1971 annual meeting of the National Reading Conference, Tampa, 
Flor Ida * 
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The Word Attack component of the Design Is presently being field 
tested at the primary levels in eighteen Wisconsin and five Colorado elemen- 
tary schools. Five of the eighteen Wisconsin schools are located in the 
central-city portion of Milwaukee. These sites are referred to as disadvan- 
taged since median reading achievement is a year or more below grade level. 

The remaining Wisconsin and Colorado sites are referred to as advantaged 
schools. This paper will report on the performance of pupils in these schools 
who had experienced the program for approximately six months. 

Objectives for the Word Attack Component 

Three major objectives have been stated for the Word Attack compo- 
nent: 

Terminal Objective 



The student upon attainment of all Level D Skills will be able to at- 
tack independently, phonieally and/or structurally regular words and will 
recognize on sight all the words on the Dolch list. Children of average or 
above average ability will attain this objective at least by the end of the 
fifth year (fourth grade) in school, while others will attain this objective 
by the end of the seventh year . 



Interim Objective 

The student upon attainment of all Level C Skills will be able to at- 
tack independently, phonieally regular one syllable words and their structural 
variants, and will recognize on sight the easier 110 words an the Dolch list. 
Children of average or above average ability will attain this objective by the 
end of the fourth year in school, while others will attain this objective by 
the end of the fifth year in school. 
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Specific Objectives 

Forty-five specific behavioral objectives have been stated elsewhere 
(Otto and Askov, 1970). The student whose pupil record indicates he has at- 
tained a particular skill will perform at a mastery level (807 o or higher) on 
a subsequently administered criterion-referenced test. 

Objectives of the Evaluation 

T' ree additional objectives have been stated for the evaluation of the 
overall program. These objectives are: 

(a) to determine whether the terminal 9 interim, and spe- 
cific objectives of the program are met. (Attain- 
ment of terminal and interim objectives is meaning- 
fully assessed only in the second year of program 
operation and thereafter.) 

(b) to determine whether the Word Attack and other 
reading skills of children using the program are 
greater than those of children not using the pro- 
gram in the same schools and at the same age- grade 
levels in the year prior. 

(c) to learn how the program is implemented in a 
variety of selected sites in which leaders are 
not exposed to a standard training program and 
how local variants affect program success. 
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Preliminary data related to the attainment o£ the Word Attack terminal 
and interim objectives are presently being gathered and therefore will not 
be reported in this paper. Subjects in the field test are presently engaged 
in mastering the forty-five specific behavioral objectives. 

At this point in time, it is perhaps more crucial to examine the objec- 
tives of the evaluation. As specified earlier, attainment of the program 
objectives is assessed meaningfully only in the second year of program opera- 
tion and thereafter. The latter two evaluation objectives, performance of 
students who were involved in the program versus those who were not and pro- 
gram implementation practices, are the primary foci of this paper. 

Design of the Study 

During May, 1970, randomly selected subjects in the twenty-three field 
tests schools who had not experienced the program were administered selected 
batteries of the Wisconsin Tests of Reading Skill Development - Word Attack as 
well as three standardised reading tests. The standardized reading tests were 
the Cooperative Primary Tests , Word Analysis (1965); the Stanford Achievement 
Test, PI and FXI (Kelly, et. al. 9 1964); and the Comprehensive Tests of Basic 
Skills , Vocabulary and Comprehension (1968), To facilitate the testing, stu- 
dents were randomly assigned to sittings and ordinarily participated in only 
one test sitting. The tests which comprise each sitting are found in Appendix 
A. During May, 1971, another random sample of students was selected and par- 
ticipated in the interim testing program. 

While the unit of reference is the individual student, the unit for 
statistical analysis must be the entire grade cohort. This is necessary since 
the presence or absence of the program is experienced for all or none of the 
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cohort and the effects of the particular conditions of implementation are 
experienced by the entire participating school staff. 
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To determine whether a change in performance between the control group 
(subjects not exposed to the program) and the experimental group (subjects 
exposed to the program) can be inferred, the data were used to test the null 
hypothesis that no change had occurred. Hypotheses concerning change at each 
grade level, and within grade level, by type of measure, were tested separately. 
In other words , for a given grade level, up to three hypotheses related to no 
change on program-dependent Design subtests, program-independent Word Attack 
tests, and standardized measures of general reading ability were tested* In 
each ease, the data consisted of one or more change scores for each school in 
the field test. Each change score represented the difference between the mean 
scores for the first implementation year and the preimplementation year for a 
given measure or set of measures* (Composite scores were formed for the 
three to five Design subtests administered in a single setting.) 

In those cases for which there is but one change score per school for 
a particular grade and content type, a two-tailed t-test was used to determine 
whether the hypothesis of no change should be rejected or not rejected. If 
the hypothesis is rejected, and if the change is positive, a gain was inferred; 
if significantly negative, a loss was inferred, 

A more complex analysis was performed when two or more change scores 
were available from each school- The three- step analysis first required re- 
scaling of the change scores to take into account scale difference between 
measures (or their composites) and to give more weight to the more reliable 
measures. Each raw change score was multiplied by a constant inversely pro- 
portional to the standard deviation of the schools 1 change scores. 



5 



6 



Next, it was determined whether the amount and direction of change 
varied by measure, as rescaled. The expectation was that change would be con- 
sistent for the various measures of a given kind. To test the hypothesis, 
contrasts between changes in the various measures were firmed, and a t^ratio 
or Hotelling's statistic computed. Rejection of the hypothesis indicated 
that degree and direction of change was systematically different for the instru- 
ments, whereas from failure to reject supported the assumption of consistency 
in change . 

Depending upon the outcome of the test of the consistency hypothesis, 
the last step varied. If consistency was inferred from step two in the ana ly- 
sis, then a T test is applied to the average change score across measures to 
determine whether the raw unit change is nonzero. Descriptive statistics indi- 
cate the direction and amount of change. If change is not consistent across 

9 

measures, the change scores for all measures are analyzed by Hotelling's T 
test to determine whether a change occurred- If this test yields significant 
results, univariate tests are subsequently used to identify the measures in 
which there was change* 

Program implementation practices, the third evaluation objective, were 
ascertained by randomly selecting a sample of teachers and mailing them ques- 
tionnaires . 

Findings 

The most marked effects of the program was related to tests associated 
with the specific program objectives- Achievement of subjects enrolled in the 
program compared to those not enrolled favored the former group on more than 
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ninety percent of the objectives. This finding pertained to both advantaged 
and disadvantaged schools. 



Insert Table 1 About Here 

In addition, performance on the standardized reading test was as good 
or better for participating pupils. Mean gains of six, nine, and thirteen 
raw score points in some inner-city schools on the Cooperative Primary Test - 
Phonics Analysis were observed. Increases for other tests were more modest. 

Performance on vocabulary and comprehension measures of reading 
achievement were slightly negative for the former and negligible for the lat- 
ter. These results may be attributable to a conversion from a general word 
recognition teaching technique to a phonic approach and emphasis on word at- 
tack skills rather than on comprehension. Increased scores on both vocabulary 
and comprehension are anticipated as the field testing continues. 



Analysis of returned teacher questions revealed that about sixty per- 
cent of the teachers are responsible for instructing a single skill group, 
thirty percent for two groups, and ten percent for three groups. Approximately 
nine percent of the children in a skill group were excused before the last 
day because they had demonstrated skill mastery. Interestingly, twenty per- 
cent of the children in a skill group receive different instruction even 
though all children in a skill group demonstrate a common need. This indi- 
cates that individualization of instruction is occurring within the skill 
group , 

Most students demonstrate skill mastery by performance on a paper -and - 

pencil test; only ten percent of the students are assessed strictly by' teacher 

O 

ERIC 




8 



TABLE 1 

SUMMARY OF SIGNIFICANT RESULTS FOR 
FIRST YEAR WORD ATTACK FIELD TEST 





Disadvantaged 


Advantaged 


Grade 


WIRED 


S tand * 


WIRED 


S tand , 


1 


<.05 


None 


<,01 


<.05 d 


2 


<,01 


<,05 a 


<.01 


N.S,® 


3 


N.S. 


N.S , b 


<,01 


<.0I(neg , ) c 


4 


N.S. 


N.S. C 


None 


\ None 



Cooperative Primary Teats - Word Analysis 

^Cooperative Primary Tests - Word Analysis 
Stanford Achievement Tests - Word Meaning and Word Study Skills 

c Comprehens ive Tests of Basic Skills - Vocabulary and Comprehension 

Cooperative Primary Tests - Word Analysis 
Stanford Achievement Tests, PI - Word Reading and Word Study Skills 

e S tanf ord Achievement Tests, PII - Word Study Skills and Word Meaning 
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judgment. The majority of teachers administer tests themselves while two- 
thirds of the teachers perform their own scoring of tests. 

The first-year field test of the Word Attack component has been com- 
pleted while the second-year of the Word Attack and first-year of the Study 
Skills components has only begun. The Design, then, is in tne first phase 
of an extensive field testing program. Additional field test results will 
be reported at professional meetings as well as through the Research and 
Development Center for Cognitive Learning publications (Quilling and Otto, 
1971 , in press) . 
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APPENDIX A 
Part 1 

Schedule of Sittings Where Present Reading 
Achievement is at or Above Grade Level 









Administration Time 


Grade 


Sitting 


Test 


(In Minutes) 


1 


1 


Cooperative Primary Tests 


40 






Word Analysis 




1 


2 


Stanford Achievement PI 








Word Reading 


15 






Word Study Skills 


25 


1 


3 


Wisconsin Tests of Reading Skill Develop- 








ment - Level B 








Test 3 - Beginning Consonant Sounds 


10 , 






Test 5 - Consonant Blends 


10 






Test 6 - Rhyming Elements 


10 


1 


4 


Wisconsin Tests of Reading Skill Develop- 








ment - Level B 








Test 7 - Short Vowels 


10 






Test 10 - Contractions 


10 






Test 11 - Base Words and Endings 


10 


2 


1 


Stanford Achievement PIX 








Word Study Skills 


30 






Word Meaning 


Id 


2 


2 


Wisconsin Tests of Reading Skill Develop- 








ment - Level C 


■t A 






Test 3 - Consonant Blends 


10 






Test 4 - Long Vowel Sounds 


10 






Test 12 - Consonant Digraphs 


10 
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Part 1, Cont. 







Test 


Administration Time 


Grade 


Sitting 


(In Minutes) 


2 


3 


Wisconsin Tests of Reading Skill Develop- 
ment - Level G 








Test 5 - Vowel + r J a-f 1* a + w 


10 






Test 6 - Diphthongs 


10 






Test 16 - Synonyms and Antonyms 


10 


2 


4 


Wisconsin Tests of Reading Skill Develop- 
ment - Level D 








Test 2 - Three Letter Consonant Blends 


10 


3 


1 


Comprehensive Tests of Basic Skills 








Vocabulary 


15 






Comprehens ion 


34 


3 


2 


Wisconsin Tests of Reading Skill Develop- 
ment - Level D 








Test 2 - Three Letter Consonant Blends 


10 






Test 3 - Silent Letters 


10 






Test 4 - Syllabication 


10 






Test 5 - Accent 


10 






Test 7 - Posses sives 


10 
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Part 2 

Schedule of Sittings Where Present Reading Achievement 
Is Below Grade Level, 19/0 



Grad© 


Sitting 


Test 


Administration Time 
(In Minutes) 


i 


"1 

1 


Wisconsin Tests of Reading Skill Develop- 
ment - Level A 

Test 1 - Rhyming Words 
Test 5 - Words and Phrases 
Test 7 - Initial Consonants 


20 

15 

15 


i 


2 


Wisconsin Tests of Reading Skill Develop- 
ment - Level B 

Test 3 - Beginning Consonant Sounds 
Test 5 ~ Consonant Blends 
Test 6 - Rhyming Elements 


10 

10 

10 


i 


3 


Wisconsin Tests of Reading Skill Develop- 
ment - Level B 

Test 8 - Consonant Digraphs - sh, ch, th 
Test 9 - Compound Words 
Test 12 - Plurals 


10 

10 

5 


2 


1 


Cooperative Primary Tests 
Word Analysis 


40 


2 


2 


Wisconsin Tests of Reading Skill Develop- 
ment - Level B 

Test 3 - Beginning Consonant Sounds 
Test 5 - Consonant Blends 
Test 6 - Rhyming Elements 


10 

10 

10 


2 


3 


Wisconsin Tests of Reading Skill Develop- 
ment - Level B 

Test 7 - Short Vowels 

Test 10 - Contractions 

Test 11 - Base Words and Endings 


10 

10 

10 
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Part 2 , Cent, 



Grade 


Sitting 


t 

Test 


Adminis tra tion Time 
(In Minutes) 


2 


4 


Wisconsin Tests of Reading Skill Develop- 
ment - Level C 








Test 3 - Consonant Blends 


10 






Test 4 - Long Vowel Sounds 


10 






Test 12 - Consonant Digraphs 


10 


3 


1 


Cooperative Primary Tests 








Word Analysis 


40 


3 


2 


Stanford Achievement Tests 








Word Meaning 


12 






Word Study Skills 


30 


3 


3 


Wisconsin Tests of Reading Skill Develop® 
ment - Level C 








Test 3 * Consonant Blends 


10 






Test 4 - Long Vowel Sounds 


10 






Test 12 - Consonant Digraphs 


10 


3 


4 


Wisconsin Tests of Reading Skill Develop® 
ment - Level C 

Test 5 - Vowel a + 1* a+w 


10 






Test 6 - Diphthongs 

(D) Test 2 - Three Letter Consonant 


10 






Blends 


10 


4 


1 


Comprehensive Tests of Basic Skills 








Vocabulary 


15 






Comprehend ion 


34 


4 


2 


Wisconsin Tests of Reading Skill Develop- 
ment - Level G 








Test 5 - Vowel aH- 1, a + w 


10 






Test 6 - Diphthongs 


10 






Test 16 - Synonyms and antonyms 
(D) Test 2 - Three Letter Consonant 


10 






Blends 


10 
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Par t 2 , Cent . 
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